@ ACT Research & Policy 


JUNE 2017 


Jill Crouse is a senior research 
associate in Research Services 
specializing in postsecondary research 
and report development. 


Matt Harmston is a senior research 
associate in Research Services 
specializing in secondary research and 
report development. 


Justine Radunzel is a principal 
research scientist in Statistical and 
Applied Research specializing in 
postsecondary outcomes research and 
validity evidence for the ACT® test. 


ACT 


Validity Evidence for STEM Interest Identification 


JILL D. CROUSE, PHD, MATT HARMSTON, MA, AND JUSTINE RADUNZEL, PHD 


In 2014, ACT developed a method to identify high school students who had an interest in STEM- 
related fields. By using ACT data elements, this method provided two different types of STEM 
interest: Expressed and measured. Based on these STEM interest types, students can be assigned 
to one of four STEM interest groups: Expressed and measured, expressed-only, measured-only, or 
an interest in non-STEM majors. This brief summarizes the results of a study that assessed the 
validity of this approach. 


STEM types are assigned on the basis of self-reported information and ACT Interest Inventory 
results obtained when students register for the ACT. Students with an expressed interest indicated 
an intent to pursue a STEM-related college major and/or occupation. Students with a measured 
interest had a highest ACT Interest Inventory scale score in Science, or had a highest scale score 
in Technology and a second-highest in Science. 


A data set of 163,342 ACT-tested students who first enrolled in college between Fall 2005 and Fall 
2009 at one of nearly 50 four-year postsecondary institutions? was used to determine if students 
who were identified as having an interest in STEM actualized this interest in college. The data set 
included up to six years of post-high school college outcomes for each student. The longitudinal 
data made it possible to determine students’ college major status within three time periods. 


Specifically, did students have a STEM college major: 


1. during their first term, 
2. atsome point in their college experience, or 


3. atthe time of college graduation (within six years). 


The data set was not a nationally representative sample of ACT-tested students. However, as can 
be seen in Table 1, the distribution of STEM interest type for the study sample is very similar to that 
of the ACT-tested graduating class of 2015. 


* A more detailed description of the data set can be found in Radunzel, J., Mattern, K., & Westrick, P. (2017). 
Who will declare a STEM major? The role of achievement and interests. lowa City, |A: ACT. 
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Table 1. Percentage of STEM-Interest Type Among STEM Interested Students 


Expressed and 


Student Group Measured Expressed Only Measured Only 
Tracked Cohort 33 47 20 
2015 Graduating Class 34 48 18 


Overall Findings 

Classification of the students’ college major as STEM or non-STEM revealed a moderately strong 
association with ACT STEM interest type as revealed by the difference in the percentage of 
declared STEM majors for students with STEM interest and students without. For the three college 


time periods studied, these percentages are: 


e 46% of students with STEM interest declared a first-term STEM major, compared to 9% of 
those with interest in other majors. 

e 59% of students with STEM interest declared a STEM major sometime during their college 
careers, compared to 16% of those with interest in other majors. 

e 48% of students with STEM interest graduated with a STEM major, compared to 9% of 


those with interest in other majors. 


Providing a more nuanced look, Figure 1 illustrates that students who fell into the expressed and 
measured STEM interest group were the most likely to pursue a STEM major, regardless of where 
they were in their college experience. Among the groups with a STEM interest, students with 
measured-only interest showed the lowest percentages of STEM college majors. These students 
had ACT Interest Inventory results that pointed to a STEM field, yet did not express this interest by 
listing a planned STEM major. With extra guidance or counseling to facilitate a greater awareness 
of STEM-related fields and career opportunities, higher rates of STEM major declaration (and 


potentially, completion) could occur for measured-only students. 
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Figure 1. Percentage of students with STEM college majors by STEM interest group and time period 


Future Research 
Differences between STEM-interested students who declared a STEM major in college and STEM- 


interested students who did not declare a STEM major offer two topics for further research: 


1. STEM-interested students with STEM college majors had higher ACT subject area and 
STEM scores than STEM-interested students with non-STEM majors even when the 
cohort was restricted to students who reached college graduation. 

2. STEM-interested females declared non-STEM majors at a higher rate than male 


counterparts. 


Discovering the reasons for these differences could be beneficial in helping STEM students 


plan high school coursework and in career guidance/counseling contexts. 


Summary 

In the end, ACT’s method for identifying STEM-interested students goes a long way toward 
distinguishing which students are likely to pursue STEM-related majors in college. The different 
STEM types allow for multiple ways of facilitating student goal attainment. Further investigation of 
factors impacting classification accuracy and associated academic achievement will enhance the 


usefulness of this indicator. 


